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ABSTRACT 


Conventional  human  engineering  legibility  specifications  for 
alphanumeric  symbology  displayed  on  video  monitors  consist  of 
predictors  of  user  performance  derived  from  legibility  research 
studies.  Typical  legibility  predictors  are  symbol  size,  symbol 
luminance,  contrast  ratio,  font,  dot  matrix  dimensions,  and 
pixel  active  area.  This  report  discusses  a  number  of  signifi¬ 
cant  problems  associated  with  the  development  and  application  of 
conventional  legibility  specifications.  In  addition,  it  presents 
a  detailed  specification  for  comparative  and  objective  evalua¬ 
tion  of  video  monitor  legibility  performance  based  upon  legibil¬ 
ity  tests  developed  for  the  OSAF.  This  specification  consists 
of  criteria  for  subject  selection,  visual  environment  design, 
display  monitor  setup,  test  material  design  and  presentation, 
legibility  task  scores,  and  legibility  test  performance.  It  is 
to  be  noted  that  this  report  is  a  revision  of  an  earlier  report  .  ^ 
under  the  same  title  (DCIEM  Tech.  Report  No.  79X01)  published  in 
January  1979.'rShiie  the  specification  for  the  Legibility  Test 
ha3  not  undergone  any  major  change,  the  proposed  Minimum  Conven¬ 
tional  Legibility  Criteria  have  been  reduced  in  number.  The 
revised  report  also  includes  a  new  appendix  describing  in  detail 
a  standardized  procedure  for  measuring  CRT  symbol  luminance  and 
calculating  contrast  ratios. 
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This  report  is  a  revision  of  an  earlier  report  under  the  sane  title 
(DCIEM  Technical  Report  No.  79X01)  published  in  January  1979.  While  the 
specification  for  the  Legibility  Test  has  not  undergone  any  major  change, 
the  proposed  Minimum  Conventional  Legibility  Criteria  have  been  reduced 
in  number.  The  revised  report  also  includes  a  new  Appendix  describing  in 
detail  a  standardized  procedure  for  measuring  CRT  symbol  luminance  and 
calculating  contrast  ratios. 

DCIEM  was  tasked  by  DCSEM  (Directorate  Computer  Systems  Engineering 
and  Maintenance)  to  undertake  the  development  of  a  legibility  specifica¬ 
tion,  based  on  human  engineering  criteria,  for  display  of  alphanumeric 
symbology  on  video  monitors  to  be  implemented  in  Base  Automated  Data  Pro¬ 
cessing  Systems  (BADPS).  In  addition,  DCSEM  requested  the  development  of 
a  methodology  by  which  DCIEM  could  assist  the  BADPS  Design  Authority  in  a 
comparative  evaluation  of  candidate  video  monitors  according  to  specifi¬ 
cation  requirements. 

The  BADPS  display  requirement  is  for  a  KVDU  (Keyboard  Video  Display 
Unit)  having  a  monochromatic  video  monitor  capable  of  displaying  the  99 
alphanumeric  and  special  symbols  of  the  ASCII  96  code  set  in  an  approxi¬ 
mate  80  symbols  per  row  by  25  row  format  common  to  the  majority  of  com¬ 
mercially  available  CRT  KVDUs.  BADPS  display  applications  utilize  a 
variety  of  alphanumeric  page  formats,  including  menus,  fill-in  forms,  and 
tabular  data  displays  ,  and  implement  features  such  as  dual  level  lumi¬ 
nance,  blinking,  and  inverse  video  to  facilitate  user  discrimination  of 
selected  data  fields. 

DCSEM  expressed  concern  that  conventional  human  engineering  specifi¬ 
cations  for  electronic  displays  are  either  too  general,  resulting  in 
acceptance  of  displays  that  are  inadequate  for  the  user  in  terms  of,  for 
example,  symbol  definition  or  contrast  ratio,  or  too  restrictive,  thereby 
rejecting  displays  which  are  in  fact  suitable  for  the  proposed  applica¬ 
tion.  It  is  necessary,  therefore,  to  Implement  an  approach  to  specifica¬ 
tion  development  that  eliminates  this  problem.  DCSEM  suggested  also  that 
the  display  specification  be  independent  of  both  display  technology 
(e.g.,  CRT,  liquid  crystal)  and  vendor  Implementation  (e.g.,  screen  area 
and  symbol  size)  (1). 

The  specification  proposed  in  this  report  is  based  upon  an  examina¬ 
tion  of  human  engineering  research  data  relevant  to  the  design  of  elec¬ 
tronic  (i.e.,  CRT  and  matrix)  display  systems.  Current  electronic 
display  design  criteria  are  derived  primarily  from  studies  involving  CRT 
displays.  For  this  reason,  particular  emphasis  was  placed  upon  review  of 
matrix  (i.e.,  plasma,  electroluminescence  and  liquid  crystal  technolo¬ 
gies)  display  research. 

The  proposed  specification  is  also  based  upon  literature  pertinent 
to  legibility  testing  of  electronic  displays.  This  literature  indicates 
a  trend  away  from  the  conventional  practice  of  specifying  characteristics 
of  display  units  which  are  known  to  have  effects  on  legibility  and  which 
are  technology-dependent,  and  toward  a  method  of  determining  legibility 
based  upon  user  performance  under  various  testing  conditions  as  described 
in  the  specification  document . 


While  the  specification  proposed  herein  relies  heavily  upon  perfor¬ 
mance  criteria  and  methodologies  derived  from  the  surveyed  legibility 
research  studies,  this  report  presents  them  in  a  clear  and  organized  for¬ 
mat  that  can  be  directly  implemented  by  the  display  systems  engineer. 


CONVENTIONAL  HUMAN  ENGINEERING  SPECIFICATIONS  FOR  ELECTRONIC  DISPLAYS 

In  general,  conventional  display  specifications  contain  a  set  of 
design  criteria  based  upon  human  engineering  research,  the  majority  of 
which  relate  to  display  legibility,  quantity  of  displayed  information, 
and  perceived  flicker.  A  study  of  relevant  literature  demonstrates  that 
most  studies  have  involved  the  use  of  CRT  rather  than  matrix  displays 
(2, 3, 4, 5, 6) .  Although  a  number  of  matrix  display  technologies  are  poten¬ 
tially  available  today,  including  plasma,  electroluminescent,  and  liquid 
crystal  panels,  CRT  technology  dominates  the  electronic  display  market  by 
virtue  of  its  relatively  low  cost,  high  reliability,  high  degree  of 
software  support,  availability  of  application  programs,  and  wealth  of 
design  experience  (7).  However,  the  applicability  of  CRT  research  data 
to  matrix  display  design  is  questionable  in  many  instances  due  to  the 
inherent  differences  in  symbol  generation  technique.  This  together  with 
the  lack  of  human  engineering  data  for  matrix  displays  suggests  that  a 
specification  for  electronic  displays  which  is  independent  of  technology 
will  not  likely  be  based  solely  upon  existing  research  data. 

Several  excellent  human  engineering  reviews  pertaining  to  electronic 
display  of  alphanumeric  symbology  are  available  (Scanlan  and  Carel,  1976; 
Gould,  1968;  Buckler,  1977;  Vanderkolk  et  al,  1975).  They  provide  a  rea¬ 
sonably  complete  summary  of  the  many  variables  from  which  the  design  cri¬ 
teria  of  conventional  specifications  have  been  derived.  These  inolude 
the  following: 


a.  CRT  symbol  generation  variables; 

-  font 

-  dot  size 

-  definition  (i.e.,  dot  matrix  dimensions) 

-  subtense 

-  aspect  ratio 

-  spacing 

-  luminance  and  contrast 

b.  CRT  raster  variables; 

-  refresh  rate 

-  resolution 

-  interlace  structure 

c.  CRT  phosphor  variables; 

-  persistence 

-  spectral  response 

d.  Matrix  symbol  generation  variables; 

-  font 

-  definition 


-  subtense 

-  edge  luminance  gradient 

-  luminance  and  contrast 

-  emitter  size 

-  emitter  spacing 

-  emitter  shape 

-  emitter  chromaticity 

-  percent  active  area 

(i.e.f  emitter  diameter/emitter  spacing) 

Other  factors  in  conventional  specifications  include: 

-  symbol  set 

-  symbol  format  (i.e.,  symbols/row  x  rows) 

-  anti-reflective  display  surface  treatment 

-  screen  size 

-  screen  orientation  relative  to  viewing  angle 

-  highlighting  (i.e.,  inverse  video,  blinking, 

and  dual  level  luminance) 


One  of  the  major  problems  encountered  by  the  specification  designer 
is  deciding  which  of  these  variables  should  be  included  in  the  specifica¬ 
tion  and  what  values  each  should  take.  A  number  of  design  criteria  are 
not  difficult  to  specify  because  either  the  data  are  conclusive  or  the 
effects  on  user  performance  are  well  understood.  For  example,  a  simple 
method  of  reducing  specular  glare  on  the  display  surface  and  thereby 
increasing  symbol  contrast  ratio  is  to  specify  that  the  display  device  be 
provided  with  an  anti-reflective  surface  (4).  Similarly,  screen  size  can 
be  specified  for  alphanumeric  displays  if  symbol  format  is  known  because 
symbol  dimensions  and  spacing  are  also  part  of  the  specification. 
Another  example  of  a  factor  that  is  relatively  easy  to  specify  is  refresh 
rate  for  CRT  displays.  Research  data  derived  from  temporal  modulation 
transfer  functions  for  the  human  visual  system  strongly  indicate  to  the 
display  designer  that  generation  of  a  standard  ,  non-interlaced  raster  on 
a  short  persistence  phosphor  such  as  P31  or  P4,  for  example,  for  display 
of  symbology  in  ambient  illumination  conditions  of,  say,  100  fc,  will 
require  a  refresh  rate  of  about  60  Hz  (2,8). 

However,  other  criteria  are  more  difficult  to  specify  because  dif¬ 
ferent  studies  appear  to  provide  conflicting  data.  For  example,  Gould 
(1968)  suggested  that  symbols  defined  by  5x7  dot  matrices  are  marginally 
acceptable  because  a  number  of  studies  had  demonstrated  the  need  for  10 
scan  lines  per  symbol  height  on  television  displays.  Vanderkolk  et  al 
(1975)  and  Scanlan  and  Carel  (1976)  concluded,  however,  that  for  static 
upright  symbols  a  5x7  matrix  with  a  relatively  high  percent  active  area 
is  sufficient  for  good  legibility  performance.  Another  example  of  data 
which  appear  to  conflict  across  studies  are  those  relating  to  symbol  sub¬ 
tense.  Some  studies  recommend  a  minimum  symbol  subtense  of  12'-15'  of 
visual  arc  for  operational  viewing  distances  while  another  study  recom¬ 
mends  a  minimum  subtense  of  22 '-26'  (3).  This  is  a  difference  of  almost 
one  hundred  percent.  Contrast  ratio  is  yet  another  example.  Gould 
(1968)  specifies  a  minimum  acceptable  contrast  ratio  of  15:1  for  CRT  sym¬ 
bology  while  Buckler  (1977)  accepts  a  minimum  contrast  ratio  of  8.5:1. 


Not  only  is  there  some  uncertainty  about  criterion  levels  to  be 
stated  in  the  specification,  but  there  is  uncertainty  regarding  the  rela¬ 
tive  importance  of  the  conventional  predictors  of  display  performance. 
For  example,  experimental  research  by  Vanderkolk  et  al  (1975)  determined 
that  four  variables  account  for  over  50  percent  of  the  legibility  of 
matrix  displays.  In  descending  order  of  importance  they  are  percent 
active  area,  symbol  subtense,  symbol  contrast,  and  surround  illumination. 
Scanlan  and  Carel  (1976)  argue,  however,  that  although  reducing  percent 
active  area  to  a  low  level  tends  to  reduce  the  legibility  of  alphanumeric 
symbols,  its  relative  importance  may  have  been  overestimated  in  the  study 
by  Vanderkolk  er  al. 

Kinney  (1966)  presents  a  succinct  discussion  of  the  problems  associ¬ 
ated  with  the  development  of  conventional  display  specifications.  In  sum¬ 
mary,  they  are: 

1.  Definitions  of  legibility  vary  from  study  to  study  with  the  result 
that  different  investigators  use  different  performance  measures.  Thus, 
the  data  cannot  be  meaningfully  compared  across  studies  with  much  confi¬ 
dence  . 

2.  Criteria  in  a  conventional  specification  are  predictors  of  display 
performance.  It  is  predicted  that  the  display  will  have  high  legibility 
if  its  symbols  satisfy  the  specified  criteria.  'Everyone's  attention  is 
directed  away  from  human  performance  and  towards  the  details  of  its  pred¬ 
ictors  . ' 

3.  'The  relationship  between  the  usually  listed  predictors  and  the  even¬ 
tual  human  performance  (in  terms  of  legibility)  is  not  technically  sound 
enough  to  warrant  placing  one's  faith  entirely  on  the  prediction  pro¬ 
cess  ' . 

M.  The  process  of  deciding  whether  or  not  a  particular  display  complies 
with  the  specification  depends  on  measurements  of  such  variables  as  dot 
size,  symbol  height,  and  contrast  ratio.  These  measurements  are  not  only 
difficult  to  make  but  are  usually  quite  imprecise. 

5.  Different  display  technologies  require  different  specifications 
because  symbol  generation  techique  is  technology-dependent.  Thus,  the 
quest  for  a  standard  display  specification  is  made  very  difficult. 

6.  Lastly,  the  display  may  be  accepted  before  legibility  is  tested 
because  the  Design  Authority  often  places  unwarranted  faith  in  the  vendor 
to  provide  a  display  which  meets  the  specification.  However,  as  is  the 
case  with  most  commercially  available  computer  terminals,  the  technical 
specification  provided  by  the  vendor  pertaining  to  symbol  features  may  be 
somewhat  suspect.  It  is  necessary,  therefore,  to  develop  other  methods 
for  specifying  electronic  display  interfaces  which  are  less  dependent 
upon  the  interpretation  of  human  engineering  data  relevent  to  predictors 
of  user  performance  and  more  directly  related  to  user  performance  per  se. 
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LEGIBILITY  TESTING 


The  use  of  a  conventional  display  specification  consisting  of  a 
minimum  set  of  well  validated  criteria  is  potentially  useful  as  a  screen¬ 
ing  medium  to  eliminate  video  monitors  that  are  clearly  unacceptable  for 
alphanumeric  symbol  display.  However,  problems  inherent  in  conventional 
specifications  that  were  discussed  in  the  previous  section  can  be  over¬ 
come  by  implementing  another  form  of  specification  based  upon  legibility 
test  and  performance  criteria. 


A  legibility  test  is  a  formal  test  in  which  each  of  a  small  number 
of  potential  users  is  required  to  detect  and  identify  symbols  presented 
on  a  display  surface  under  rigorously  controlled  test  conditions.  A  set 
of  candidate  video  monitors  can  be  comparatively  evaluated  using  a  legi¬ 
bility  test  approach  because  each  subject  in  the  test  performs  with  each 
monitor  under  identical  and  highly  controlled  conditions. 


The  literature  on  legibility  testing  demonstrates  that  the  technique 
is  an  extremely  useful  one  and  that  it  can  be  easily  applied  to  any 
display  technology  (5,9,10,11).  The  emphasis  is  directly  upon  user  per¬ 
formance  as  measured  with  the  actual  display  and  not  upon  the  predictors 
of  user  performance  which  comprise  conventional  specifications  for  elec¬ 
tronic  displays.  Thus  the  problems  of  validity  of  specification  and 
measurement  of  display  characteristics  such  as  emitter  3ize  and  luminance 
are  eliminated. 


There  are  three  types  of  user  tasks  commonly  used  in  legibility 
testing;  namely,  tachistoscopic  recognition,  visual  search,  and  reading 
tasks . 


Tachistoscopic  recognition  tasks  have  been  used  by  several  investi¬ 
gators  and  consist  of  sequential  presentation  of  single  symbols  to  the 
subject  for  recognition,  usually  under  conditions  that  significantly 
degrade  symbol  legibility  (3,5).  Each  symbol  is  randomly  selected  from  a 
symbol  set  and  displayed  for  a  brief  period  of  time,  of  the  order  of  tens 
of  milliseconds,  at  a  fixed  location  on  the  display  surface.  The 
subject's  task  is  to  call  out  the  name  of  the  symbol  as  quickly  as  possi¬ 
ble.  Recognition  times  and  naming  errors  are  assumed  to  be  directly 
related  to  symbol  legibility.  Subjects  are  presumed  to  quickly  and  accu¬ 
rately  recognize  symbols  that  are  legible. 

Visual  search  tasks  have  also  been  utilized  in  legibility  testing 
(12).  The  subject  is  presented  with  a  target  symbol  or  word  followed  by 
an  array  of  symbols  or  words  on  the  display  surface.  It  is  common  prac¬ 
tice  to  use  small  nonsense  words  such  as  WBZ  or  SQB  rather  than  meaning¬ 
ful  words  such  as  DOG  or  BAT  in  order  to  ensure  that  the  subject  is 
searching  each  symbol.  The  subject's  task  is  to  find  the  target  symbol  or 
word  amongst  the  displayed  array  as  quickly  as  possible.  Again,  search 
time  is  taken  to  be  a  measure  of  symbol  legibility. 

Reading  tasks  are  the  most  common  among  established  legibility  tests 
for  the  simple  reason  that  subject  behaviour  (i.e.,  verbal  responses)  is 
readily  apparent  and  relatively  easy  to  control  and  measure  (10,12,13). 
For  these  reasons  the  reading  task  has  been  selected  as  the  type  of  legi¬ 
bility  task  to  be  implemented  in  the  BADPS  display  monitor  specification. 
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In  legibility  testing,  the  onus  is  upon  the  Design  Authority  to 
specify  the  test  criteria  and  test  conditions.  Thus  it  is  necessary  to 
discuss  the  main  components  of  legibility  test  and  performance  specifica¬ 
tions.  They  can  be  categorized  as  follows: 


1.  Subject  selection  criteria; 

2.  Environmental  criteria; 

3-  Display  device  setup  criteria; 

4.  Test  material  criteria; 

5.  Array  presentation  criteria; 

6.  Legibility  task  criteria; 

7.  Legibility  scores  and  performance  criteria. 


J..  Subject  Selection  Criteria 

Each  subject  must  be  screened  for  visual  abnormalities  in  near  and 
far  acuity,  phoria  (i.e.,  balance  of  the  extra-ocular  muscles  of  the 
eye),  colour  vision,  and  depth  perception  using  a  standard  vision  tester. 
Those  subjects  with  corrected  vision  would  be  expected  to  wear  their  eye 
glasses  or  contact  lens  during  the  vision  tests. 

Subjects  should,  if  at  all  possible,  be  randomly  selected  from 
amongst  the  population  of  potential  users  of  the  display  system. 

2_.  Environmental  Criteria 

The  display  devices  should  be  evaluated  under  environmental  condi¬ 
tions  similar  to  those  in  the  operational  environment  in  which  they  will 
be  used.  For  BADPS  applications,  it  is  suggested  that  ambient  illumina¬ 
tion  levels  approximate  those  levels  found  in  modern  office  environments, 
(i.e. ,  50-100  fc.) 

The  display  surface  of  the  monitor  should  be  shielded  from  direct 
sources  of  illumination  in  order  to  minimize  specular  glare. 

The  auditory  environment  should  be  strictly  controlled  such  that  it 
is  free  of  any  potentially  distracting  sources  of  noise.  Overall  sound 
pressure  levels  should  be  no  higher  than  55  dbA. 

It  is  essential  that  each  display  device  be  tested  under  identical 
environmental  conditions. 

•3.*  Display  Device  Setup  Criteria 

For  comparative  evaluation  of  BADPS  candidate  video  monitors  it  will 
be  necessary  to  standardize  the  setting  up  of  each  monitor.  The  setup 
procedure  must  address  the  problems  of  symbol  and  background  luminance. 

JL*  Te3t  Material  Criteria 

The  legibility  test  specification  must  include  a  description  of  the 
symbol  set,  symbol  display  format,  and  symbol  selection  criteria.  With 
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reading  tasks,  it  is  common  to  divide  the  94  ASCII  symbols  into  the  62 
alphanumeric  symbols  (i.e.,  A-Z,  a-z,  0-9)  and  the  32  special  symbols 
(i.e.,  !"#$?&  etc.).  Symbols  are  randomly  selected  such  that  each  symbol 
has  an  equiprobable  chance  of  display. 

Array  Presentation  Criteria 

These  criteria  describe  the  temporal  and  spatial  aspects  of  symbol 
presentation.  Spatial  criteria  specify  viewing  distances  and  screen 
locations  at  which  the  symbol  arrays  are  to  be  displayed.  Temporal  cri¬ 
teria  specify  display  schedules  for  arrays  within  a  single  testing  trial 
and  specify  subject  rest  periods.  Symbols  are  displayed  in  an  array  for¬ 
mat  (e.g.,  3  rows  x  4  symbols  per  row)  for  most  reading  tasks  in  legibil¬ 
ity  testing. 

_6.  Legibility  Task  Criteria 

These  criteria  specify  the  control  and  verbal  responses  required  of 
the  subject  during  the  running  of  the  legibility  test.  Criteria  for 
design  of  the  instructions  and  practice  trials  which  are  given  to  the 
subject  may  also  be  specified. 

Implementation  of  the  legibility  task  requires  the  use  of  a  host 
computer  and  medium  speed  (e.g.,  2400  Baud)  data  link  with  the  video  mon¬ 
itor  under  test . 

The  computer  performs  the  functions  of: 

-  displaying  instructions  to  the  subject; 

-  selecting  and  displaying  alphanumeric  test  arrays; 

-  timing  control  responses  given  by  the  subject; 

-  computing  and  storing  session  performance  statistics. 

Three  additional  pieces  of  hardware  required  for  a  reading  type  of 
legibility  test  are  a  tape  recorder  for  recording  the  subject's  verbal 
responses,  a  microphone,  and  a  small  handheld  pushbutton  control  with 
which  the  subject  controls  the  display  of  each  array. 


£.  Legibility  Scores  and  Performance  Criteria 

The  raw  data  from  tachistoscopic  and  reading  types  of  legibility 
tests  consist  of  recognition  times  and  recognition  errors.  Inherent  in 
the  concept  of  'recognition  time'  is  the  time  required  to  translate  the 
output  of  the  perceptual  processes  to  the  appropriate  motor  response 
and, hence,  recognition  time  is  called  reading  time. 

The  raw  data  from  reading  tasks  consist  of  the  following: 

-  symbol  array  reading  times  as  monitored 
by  the  host  computer;  and 

-  tape  recordings  of  the  subject's  verbal  responses. 

From  these  data  are  computed  symbol  reading  rates  and  reading  errors. 
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Uncorrected  rend  ing  rate  is  defined  as  the  number  of  correct  and  * 

incorrect  naming  responses  per  minute. 

Corrected  reading  rate  is  defined  as  the  number  of  correct  naming 
responses  per  minute. 

A  reading  error  is  defined  to  occur  whenever  the  subject  incorrectly  ® 

names  a  jisplayed  symbol  (i.e.,  error  of  commission)  or  neglects  to  name 
a  displayed  svmbc.  (i.e.,  error  of  omission). 

Reading  error  in  defined  as  the  percentage  of  displayed  symbols  that 
are  read  in  error . 

• 

Reading  errors  f'an  be  arranged  in  what  is  called  a  'confusion' 
matrix  (5,9,11).  Suppose,  for  example,  that  a  symbol  set  consists  of 
four  symbols.  A,  B,  2,  and  8,  and  that  each  symbol  was  presented  to  each 
of  five  subjects  10  times  for  a  given  monitor.  Thus  each  symbol  was 
presented  50  times  during  the  course  of  the  legibility  test.  Further, 
assume  that  all  the  As  and  2s  were  correctly  named  and  that  the  B  was  * 

confused  with  the  8  five  times  and  with  the  A  five  times.  Assume  also 
that  the  8  was  confused  with  the  A  six  times  and  with  the  B  four  times. 

The  confusion  matrix  representing  these  data  would  appear  as  shown  in 
Figure  1 . 


SYMBOL 

NAMED 


SYMBOL 

DISPLAYED 

A 

B 

2 

8 

A 

50 

5 

0 

6 

B 

0 

40 

0 

4 

2 

0 

0 

50 

0 

8 

0 

5 

0 

40 

Fig.  1.  Raw  data  from  a  legibility  test  arranged 
in  a  confusion  matrix. 


X  ! 

The  confusion  matrix  for  each  video  monitor  i3  used  to  compute  five 
legibility  scores.  These  scores  have  been  adapted  from  the  legibility 
test  literature  (5,10)  and  are  defined  as  follows: 


O 


_L-  Corrected  Reading  Rate : 

total  correctly  named  symbols 


CRR  = 


2_.  Reading  Error : 


RE  = 


total  reading  time 


total  incorrectly  read  symbols 


x  60  symbol s/min 


total  symbols  displayed 


x  100  % 


1- 


Reading  Error  per  Symbol :  (This  3core  is  computed  for  the  symbol 
having  the  greatest  number  of  read ing  errors . ) 


RES  = 


total  number  of  times  symbol  incorrectly  read 
total  symbols  displayed 


x  100  * 


4..  Confusion  Error  per  Symbol :  (This  score  is  computed  for  the  symbol 

having  the  greatest  number  of  errors  in  a  single  cell  of 
the  confusion  matrix. ) 


CES  = 


total  number  of  times  symbol  confused 
(i.e.,  single  cell  score) 


total  symbols  displayed 


x  100  % 


For  example,  from  Fig.  1: 

RE  =  (20/200)  x  100  %  =  10.0  % 

RES  =  (10/200)  x  100  %  =  5.0  ?  (Both  'B'  and  '8'  have  10 
errors . ) 

CES  =  (6/200)  x  100  %  -  3-0  %  (The  symbol  '8'  has  the  greatest 
number  of  errors  (6)  in  a  single  cell  of  the  confusion  matrix.) 

If  the  total  reading  time  had  been  90  seconds,  then 

CRR  =  (180/90)  x  60  =  120  symbols  per  min 


5..  Information  Transfer  Rate  : 

Information  Transfer  Rate  is  a  composite  measure  of  legibility  per¬ 
formance  derived  from  information  theory  concepts  as  they  have  been 
applied  in  the  discipline  of  mathematical  psychology.  (Garner,  1962)  Its 
composite  nature  permits  one  to  measure  the  actual  amount  of  information 
which  the  display  transmits  to  the  user  by  combining  both  rate  and  error 
scores . 
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ITR  =  |log4N  -  l  p(y) .log^p(y)  *  H  p(x,y) .log2p(x,y) 

x  [  total  symbols  displayed  1  bits/second 

|  total  reading  time  I 

where  N  =  size  of  symbol  set 

p(y)  =  probability  of  yth  symbol  being  named 

p(x,y)  =  probability  associated  with  row  y  and 
column  x  in  the  confusion  matrix 


i 


f 


* 


ITR  can  be  computed  from  the  data  presented  in  Fig.  1  as  follows: 

Step  1 . 

Transform  the  data  in  each  cell  of  the  confusion  matrix  to  cell 
probabilities  as  shown  in  Figure  2. 

Step  2. 

Add  the  probability  data  for  each  row  to  produce  a  p(y)  for 
each  symbol  in  the  symbol  set  as  shown  in  Figure  2. 


i. 


SYMBOL 

NAMED 


SYMBOL  DISPLAYED 


A 

B 

2 

8 

A 

.250 

.025 

0 

.030 

B 

0 

.200 

0 

.020 

2 

0 

0 

.250 

0 

0 

.025 

0 

.200 

! 


I 


t 


Fig.  2.  Raw  data  from  fig.  1  transformed  to 

probability  data  in  a  confusion  matrix. 


Step  3. 


Compute  logjN.  For  the  example  given,  N  =  4. 
Therefore,  log2N  =  2. 


Step  4. 

Compute  p(y).log2p(y)  for  each  p(y)  and  p(x,y) .loga p(x,y) 
for  each  p(x,y).  Note  that  each  computation  yields  a 
negative  number. 

For  the  example  given,  the  following  data  result.  They  were 
generated  using  a  table  from  Ref.  14  (pp.  345-6)  and, 
where  necessary,  linear  interpolation. 


P  ( x ,  y ) 

p(x,y).lcg2 

0.250 

-0.5000 

0.025 

-0. 1324 

0.030 

-0. 1518 

0.200 

-0.4644 

0.020 

-0.1129 

0.250 

-0.5000 

0.025 

-0.1324 

0.200 

-0.4644 

p(y) 

p(y).log2: 

0.305 

-0.5225 

0.220 

-0.4806 

0.250 

-0.5000 

0.225 

-0.4842 

I  =  -1.9873 


ll  =  -2.4583 


Step  4.  r 

Compute  I log2 N  -  l  p(y ) .log2 p(y )  +  H  p(x,y) ,log2 p(x,y) . 

For  the  example  given,  this  value 


=  2  -(-1.9873)  +(-2.4583) 

=  2  +  1.9873  -  2.4583 
=  +1.5290  bits/symbol 

Note  that  this  value  will  always  be  less  than  log2N. 


Step  5. 

Compute  ITR. 


For  the  example  given: 

ITR  =  +1.5290  bits/symbol  x  (200/90)  symbols/sec 


=  3.40  bits/sec 
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PROPOSED  LEGIBILITY  SPECIFICATION  £01?  VIDEO  MONITORS 

The  following  legibility  specification  is  divided  into  three  sec¬ 
tions: 

A.  Minimum  Conventional  Legibility  Criteria 

B.  Legibility  Test  Performance  Criteria 

C.  Legibility  Test  Specification 


The  Minimum  Conventional  Legibility  Criteria  are  derived  from  legi¬ 
bility  research  data  applicable  to  electronic  displays.  These  criteria 
are  provided  to  enable  the  quick  rejection  of  candidate  video  monitors 
which  do  not  meet  what  are  considered  to  be  minimal  human  engineering 
standards  for  legibility  of  CRT-displayed ,  dot  matrix  symbology. 

The  Legibility  Test  Performance  Criteria  enable  the  acceptance  or 
rejection  of  those  monitors  which  satisfy  the  criteria  in  Part  A  of  the 
specification.  Those  monitors  which  pass  the  Legibility  Test  are  then 
rank-ordered  in  terms  of  their  ITR  scores. 

The  Legibility  Test  Specification  describes  in  detail  how  the  legi¬ 
bility  test  is  to  be  run,  controlled  and  scored. 


;*•»* —  •. - 
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Part  A:  MINIMUM  CONVENTIONAL  LEGIBILITY  CRITERIA 

1.  Minimum  Symbol  Height: 

3.1  mm  (based  upon  minimum  acceptable  angular 

subtense  of  18'  and  60  cm  viewing  distance) 

Symbol  height  is  defined  as  the  centre-to-centre  distance 
between  the  top  and  bottom  pixels  as  shown  below. 


Minimum  symbol  height 
=  3. 1  mm 


2.  Minimum  Symbol  Matrix  Dimensions: 

5x7 

3.  Minimum  Symbol  Luminance: 

25  ftL  (see  Appendix  A  for  measurement  procedure) 
*1.  Minimum  Contrast  Ratio: 

4:1  (see  Appendix  A  for  calculation  procedure) 

with  ambient  Illumination  in  range  75-100  fc. 


5.  Minimum  Pixel  Diameter  to  Pixel  Spacing  Ratio: 

0.7 

Pixel  diameter  can  be  calculated  using  the  shrinking 
raster  method  described  in  Ref.  15.  Pixel  spacing  can 
be  calculated  from  the  data  for  symbol  height  and 
symbol  definition. 

6.  Minimum  Treatment  of  Screen  Surface: 


a  diffusing  treatment  that  effectively  reduces 
specular  glare  from  the  screen 
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Part  B:  LEGIBILITY  TEST  PERFORMANCE  CRITERIA 

Note:  The  following  criteria  apply  only  to  the  ^ 

set  of  62  alphanumeric  symbols.  They  are  based 
upon  criteria  found  in  Refs.  5  and  10. 

1.  Minimum  Acceptable  Corrected  Reading  Rate  ^ 

CRR  =  120  symbols  per  min 

2.  Maximum  Acceptable  Reading  Error 

RE  =  3  t  1 

3-  Maximum  Acceptable  Reading  Error  per  Symbol 
RES  =  0.45  %  (i.e. ,  15  %  of  RE) 

4.  Maximum  Acceptable  Confusion  Error  per  Symbol  * 

CES  =  0.30  it  (i.e.,  10  %  of  RE) 

5.  Information  Transfer  Rate 


ITF  is  computed  only  For  those  monitors  that  satisfy  1-4. 


I 
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Part  £:  LEGIBILITY  TEST  SPECIFICATION 


J..  Subject  Selection 

Six  subjects  shall  be  selected  for  the  legibility  test.  Each  of  the 
subjects  shall  have  20/20  far  and  normal  near  visual  acuity,  normal 
phoria,  normal  colour  vision,  and  normal  depth  perception  as  tested  using 
a  standard  vision  tester.  Subjects  who  normally  wear  eye  glasses  or  con¬ 
tact  lens  shall  wear  them  during  vision  testing  and  legibility  testing. 

Where  possible,  the  subjects  shall  be  randomly  selected  from  the 
potential  user  population  and,  ideally,  should  not  have  extensive  experi¬ 
ence  with  video  terminals. 

During  the  running  of  the  legibility  tests,  subjects  shall  wear  a 
black  matte  smock  in  order  to  reduce  reflection  of  external  light  onto 
the  screen. 


2.-  Environmental  Controls 

Environmental  conditions  shall  be  identical  from  one  testing  session 
to  another. 

The  testing  environment  shall  have  an  illumination  level  in  the 
range  75-100  fc.  as  measured  with  a  properly  calibrated  photometer. 
Fluorescent  lighting  shall  be  used . 

The  testing  environment  shall  be  free  from  distracting  sources  of 
noise.  The  overall  noise  level  shall  be  no  greater  than  55  dbA. 

The  testing  environment  shall  be  windowless. 


3.-  Video  Monitor  Setup 

A  position  for  placement  of  monitors  shall  be  selected  such  that 
specular  and  diffuse  glare  from  the  display  surface  is  reduced  to  a  rea¬ 
sonable  minimum. 

The  position  of  the  monitor  in  the  testing  environment  shall  be 
identical  for  each  monitor  under  evaluation. 

For  CRT  monitors,  background  raster  luminance  shall  be  adjusted  to 
obtain  a  'just  invisible'  condition. 

If  the  monitor  is  provided  with  an  external  'contrast'  control,  then 
symbol  luminance  shall  be  set  to  a  level  considered  optimal  in  a  pretest 
involving  4-6  experienced  CRT  display  users.  Background  raster  luminance 
shall  satisfy  the  'just  invisible'  condition. 
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If  a  CRT  monitor  is  provided  with  an  external  focus  control,  then 
symbol  focus  shall  be  adjusted  to  an  optimal  level  as  per  the  above  para¬ 
graph  . 


1*  Test  Matariri 

The  symbol  set  for  testing  purposes  shall  consist  of  the  62 
alphanumeric  symbol  set  described  in  Part  A  of  this  specification. 

The  unit  of  symbol  data  displayed  to  the  subject  shall  be  an  array 
of  dimension  4x3  (i.e.,  3  rows  of  symbols,  with  4  symbols  in  each  row). 
Horizontal  and  vertical  spacing  shall  be  the  same  as  that  provided  on  the 
operational  display. 

The  symbols  for  each  array  shall  be  preselected  by  DCIEM  such  that 
the  following  conditions  are  satisfied: 

a.  Thirty  one  arrays  shall  be  presented  for  each  monitor/test  condition 
for  a  total  of  372  (31x3x4)  symbols  to  be  named  by  the  subject  for  each 
monitor  tested; 

b.  The  62  symbols  shall  be  distributed  in  an  eauiprobable  manner  for  each 
monitor/test  condition  (i.e.,  each  of  the  62  symbols  shall  appear  6  times 
in  the  total  372  symbol  presentations  for  each  monitor/subject  test  con¬ 
dition)  . 

c.  No  symbol  shall  be  repeated  in  an  array; 

d.  The  arrays  shall  not  contain  meaningful  combinations  of  symbols  (e.g., 
LOVE,  TRAP). 

e.  The  total  number  of  sets  of  31  arrays  shall  equal  2xN,  where  N  is  the 
number  of  monitors  under  evaluation;  one  set  shall  be  used  in  practice 
sessions  while  the  other  set  shall  be  used  in  testing  sessions; 

f.  The  order  of  presentation  to  the  subject  of  arrays  and  monitors  will 
be  randomized. 


1-  Array  Rregepfratlon 

The  31  arrays  presented  in  each  subject/monitor  test  condition  shall 
be  presented  at  5  screen  positions  as  shown  in  Figure  3»  The  order  of 
presentation  of  the  arrays  shall  be  as  shown  in  Figure  3.  The  number  of 
arrays  presented  at  each  screen  position  shall  be  6  except  for  the  centre 
position  which  shall  have  7  arrays  (i.e.,  6+6+6+6+7=31 ) • 

Screen  positions  are  defined  as  follows: 


Position  #1 

Upper  left  symbol  of  the  array  in  the  upper  left 
(i.e.,  home)  position  of  the  screen  symbol  format; 

Position  #2 

Upper  right  symbol  of  the  array  in  the  upper  right 
position  of  the  screen  symbol  format; 

Position  #3 

Lower  right  symbol  of  the  array  in  the  lower  right 
position  of  the  screen  symbol  format; 

Position  #4 

Lower  left  symbol  of  the  array  in  the  lower  left 
position  of  the  screen  symbol  array; 

Position  #5 

Symbol  array  centred  on  screen  symbol  format . 


screen  position 

number  of  arrays 
at  position 


Figure  3.  Array  presentation  protocol  for  the  proposed 
BADPS  legibility  test. 


For  a  single  testing  session  in  which  the  subject  will  name  the  sym¬ 
bols  in  31  arrays,  the  first  6  arrays  will  be  presented  sequentially  at 
screen  position  #1  followed  by  6  arrays  at  position  #2  and  so  on. 
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The  arrays  presented  at  each  screen  position  shall  be  presented  one 
after  the  other  with  no  blanking  interval  between  presentations.  The 
subject  shall  control  the  presentation  of  each  array  by  operating  a  small 
handheld  pushbuton  control. 

The  subject  shall  control  the  onset  of  presentation  of  the  first 
array  at  each  screen  position.  That  is  to  say,  the  subject  will  be  able 
to  pause  between  screen  positions. 


4*  Sub  ject 's  Task 

The  subject's  task  shall  be  an  array  reading  task. 

The  instructions  for  the  subject's  task  shall  be  written  by  DCIEM 
and  shall  meet  established  criteria  for  task  instruction  design. 

Each  test  session  shall  be  preceded  by  a  practice  session. 

The  subject  shall  be  comfortably  seated  directly  in  front  of  the 
test  monitor  at  a  viewing  distance  of  90  cm  from  the  centre  of  display 
screen.  A  microphone  shall  be  positioned  in  close  proximity  to  the 

subject's  mouth  in  order  to  prevent  the  subject  from  leaning  towards  the 
screen . 

When  the  subject  operates  the  pushbutton  control  initially,  the  word 
'START'  shall  be  displayed  at  screen  position  #1.  When  the  subject 
operates  the  control  a  second  time,  the  first  array  shall  be  displayed  at 
position  # 1 . 

The  subject  shall  then  attempt  to  name  the  symbols  in  the  array. 

When  this  task  ha3  been  completed,  the  subject  shall  again  operate  the  PB 
which  shall  cause  the  next  array  to  be  displayed  and  so  on. 

The  subject  shall  name  the  symbols  in  a  normal  reading  fashion, 

namely,  from  left  to  right  and  top  to  bottom.  The  subject's  verbal 
responses  shall  be  recorded  on  a  tape  recorder. 

After  the  6th  array  at  screen  position  #1  has  been  named,  the  word 
'STOP'  shall  be  displayed  at  position  #1  and  the  word  'START'  shall  be 
displayed  at  position  #2. 

The  subject  may  now  pause  before  operating  the  PB  again.  Once  the 
PB  is  operated,  however,  the  above  sequence  of  events  is  repeated. 

The  computer  shall  log  the  time  of  display  of  the  first  array  and 

the  time  at  which  the  last  array  is  erased  for  each  screen  position.  The 

reading  time  for  that  screen  position  shall  be  the  difference  between  the 
two  times .  H 

The  subject  shall  not  be  informed  of  the  correctness  of  incorrect¬ 
ness  of  his/her  responses  until  all  the  data  are  collected  for  that  sub¬ 
ject  . 
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APPENDIX 


A  STANDARDIZED  PROCEDURE  FOR  MEASURING  CRT  SYMBOL  LUMINANCES 

- m  mm7n7N(r  -coNTOsn^rrus - 


A.  Photometric  Equipment 

1.  Components: 

Photometric  microscope 
Microscope  stand 
Photometer 

2.  The  photometric  system  components  for  measuring  CRT-displayed  dot 
luminance  should  have  technical  and  performance  specifications  at 
least  equivalent  to  those  of  the  Gamma  Scientific  2400  photometer 
amd  model  700-10  photometric  microscope.  In  particular,  the  area  of 
acceptance  of  the  photometric  probe  as  measured  at  the  screen  must 
be  smaller  than  the  CRT  dot  area  as  viewed  through  the  photometric 
microscope  in  order  to  achieve  valid  results. 

3.  The  photometric  system  shall  have  been  calibrated  according  to  the 
manufacturer's  instructions  using  an  NBS  traceable  source. 


B.  Test  Environment 

1.  In  order  to  obtain  valid  contrast  ratios  the  illumination  level  in 
the  test  environment  must  approximate  that  in  the  operational 
environment.  To  ensure  constancy  of  illumination  level  the  test 
environment  must  be  windowless. 

2.  In  addition,  the  tester  should  wear  a  matte  black  smock,  especially 
If  the  luminance  measures  are  to  be  read  from  the  photometer's 
display  while  standing  in  close  proximity  (ie.,  6  ft  or  less)  to  the 
test  screen.  The  tester  should  stand  at  least  6  ft  from  the  screen 
if  at  all  possible  to  prevent  light  energy  reflected  by  the  tester 
from  having  any  significant  effect  on  symbol  and  background  lumi¬ 
nances. 


C.  CRT  Video  Monitor  Setup 

1.  Orient  the  screen  as  it  would  be  oriented  In  the  operational 
environment.  Ensure  that  specular  reflections  from  the  screen  are 
reduced  as  much  as  possible  given  the  illumination  system  In  the 
test  environment. 

2.  Set  symbol  luminance  to  a  level  considered  optimal  in  a  pretest 
Involving  a  small  number  (3-6)  of  experienced  CRT  terminal  users. 
Maintain  raster  background  luminance  at  a  ’just  invisible’  level. 


3.  Display  a  full  screen  of  symbols  ( ' Hs * )  for  at  least  one-half  hour 
prior  to  taking  any  luminance  measurements.  This  will  ensure  that 
the  display  circuitry  is  warmed  up  and,  therefore,  stable. 


0.  Test  Procedure 

1.  Display  the  * symbol  in  the  centre  and  corner  symbol  cells  on  the 
screen  as  shown  in  Fig.  1. 


perimeter  of  screen 
perimeter  of  raster 


symbol  cell 


Fig.  1.  Symbol  cell  locations  for  luminance 
measurement  procedure. 


2.  Position  the  photometric  microscope  over  one  of  the  five  displayed 
syntools  according  to  the  manufacturer's  instructions.  Ensure  that 
the  microscope's  fibre  probe  is  positioned  over  the  centre  of  the 
symbol  and  that  the  diameter  of  the  area  of  acceptance  of  the  probe 
at  the  screen  as  viewed  through  the  microscope  is  significantly 
smaller  than  symbol  diameter,  (see  Fig.  2) 


area  of  acceptance  of  probe 
CRT  pixel 


Fig.  2.  Size  and  location  of  probe  area  of 
acceptance  relative  to  area  of 
displayed  symbol. 


3.  Record  symbol  luminance,  (see  Fig.  4) 

4.  Erase  the  symbol  and  record  background  luminance. 


A-3 


5.  Position  the  microscope  over  one  of  the  remaining  symbols  and  repeat 
steps  3  and  4. 

6.  Repeat  steps  2-4  until  luminance  measures  for  all  five  symbols 

have  been  recorded. 

7.  Display  the  *H*  symbol  in  the  five  symbol  cells  shown  in  Fig.  1. 

8.  Repeat  steps  2-6  until  luminance  measures  for  all  five  *H '  symbols 
have  been  recorded.  Ensure  that  the  probe  is  positioned  over  the  'H1 
symbol  as  shown  in  Fig.  3. 


area  of  acceptance  of  probe 


Fig.  3.  Size  and  location  of  probe  area  of 
acceptance  relative  to  area  of 
displayed  symbol  'H'. 


E.  Contrast  Ratio  Calculation 

1.  Overall  contrast  ratio  for  the  monitor  is  calculated  according  to 
the  following  equation: 

10 

CR  -  0.1  x  l  (Ls  /  Lh  ) 
i-1 

where  CR  =  overall  contrast  ratio 

L  =  luminance  with  symbol  displayed 
L  luminance  with  symbol  erased 
i  =  measurement  number 


*  I 
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CRT  SYMBOL  LUMINANCE-CONTRAST  RATIO  DATA 


Date: _  Location: _ 

Illumination  Level:. _ Tester: 


I 


Measurement 

No. 

Symbol 

Position 

L 

a 

B 

L  /  L 
e  b 

1 

• 

C 

2 

• 

ULC 

3 

• 

URC 

4 

• 

LLC 

5 

• 

LRC 

6 

H 

C 

7 

H 

ULC 

8 

H 

URC 

9 

H 

LLC 

10 

H 

LRC 

TOTAL  = 

CONTRAST  RATIO  =  O.i  x  TOTAL  = 


Fig.  4.  Table  for  recording  CRT  luminance  data 


